A Frequency Analysis of Monte-Carlo and other Numerical Integration Schemes

The numerical calculation of integrals is central to many computer graphics algorithms such as Monte-Carlo Ray Tracing. We show that such methods can be studied using Fourier analysis. Numerical error is shown to correspond to aliasing and the link between properties of the sampling pattern and the integrand is studied. The approach also permits the unified study of image aliasing and numerical integration, by considering a multidimensional domain where some dimensions are integrated while others are sampled

Controllable computer graphics for compelling depiction and animation

A full-length feature film such as Pixar's Toy Story, or the award-winning educational programs such as Walking with Dinosaurs, require a production time of several years and draw on the full-time efforts of several hundred skilled employees. Digital videography and photography has equally broad impact as everyone uses photos and videos to record memories of friends, family and events. Despite a wealth of sophisticated techniques for manipulating photographs, illustrations, and motions the compelling images in educational videos and feature films are more commonly the results of artistry and of painstaking work than of intuitive tools. As a result, despite their potential to revolutionize all educational material, high quality visual aids are used infrequently because of the extensive production costs. In the MIT Computer Graphics Group, we evolve these techniques to make them more accessible to inexperienced authors: scientists, educators, storytellers, and other broad public. We present easy-to-use tools that reduce the cost of producing compelling photographs, illustrations, and motions.Singapore-MIT Alliance (SMA

A Benchmark of Computational Models of Saliency to Predict Human Fixations

Many computational models of visual attention have been created from a wide variety of different approaches to predict where people look in images. Each model is usually introduced by demonstrating performances on new images, and it is hard to make immediate comparisons between models. To alleviate this problem, we propose a benchmark data set containing 300 natural images with eye tracking data from 39 observers to compare model performances. We calculate the performance of 10 models at predicting ground truth fixations using three different metrics. We provide a way for people to submit new models for evaluation online. We find that the Judd et al. and Graph-based visual saliency models perform best. In general, models with blurrier maps and models that include a center bias perform well. We add and optimize a blur and center bias for each model and show improvements. We compare performances to baseline models of chance, center and human performance. We show that human performance increases with the number of humans to a limit. We analyze the similarity of different models using multidimensional scaling and explore the relationship between model performance and fixation consistency. Finally, we offer observations about how to improve saliency models in the future

A Gaussian Approximation of Feature Space for Fast Image Similarity

We introduce a fast technique for the robust computation of image similarity. It builds on a re-interpretation of the recent exemplar-based SVM approach, where a linear SVM is trained at a query point and distance is computed as the dot product with the normal to the separating hyperplane. Although exemplar-based SVM is slow because it requires a new training for each exemplar, the latter approach has shown robustness for image retrieval and object classification, yielding state-of- the-art performance on the PASCAL VOC 2007 detection task despite its simplicity. We re-interpret it by viewing the SVM between a single point and the set of negative examples as the computation of the tangent to the manifold of images at the query. We show that, in a high-dimensional space such as that of image features, all points tend to lie at the periphery and that they are usually separable from the rest of the set. We then use a simple Gaussian approximation to the set of all images in feature space, and fit it by computing the covariance matrix on a large training set. Given the covariance matrix, the computation of the tangent or normal at a point is straightforward and is a simple multiplication by the inverse covariance. This allows us to dramatically speed up image retrieval tasks, going from more than ten minutes to a single second. We further show that our approach is equivalent to feature-space whitening and has links to image saliency

User-assisted intrinsic images

For many computational photography applications, the lighting and materials in the scene are critical pieces of information. We seek to obtain intrinsic images, which decompose a photo into the product of an illumination component that represents lighting effects and a reflectance component that is the color of the observed material. This is an under-constrained problem and automatic methods are challenged by complex natural images. We describe a new approach that enables users to guide an optimization with simple indications such as regions of constant reflectance or illumination. Based on a simple assumption on local reflectance distributions, we derive a new propagation energy that enables a closed form solution using linear least-squares. We achieve fast performance by introducing a novel downsampling that preserves local color distributions. We demonstrate intrinsic image decomposition on a variety of images and show applications.National Science Foundation (U.S.) (NSF CAREER award 0447561)Institut national de recherche en informatique et en automatique (France) (Associate Research Team “Flexible Rendering”)Microsoft Research (New Faculty Fellowship)Alfred P. Sloan Foundation (Research Fellowship)Quanta Computer, Inc. (MIT-Quanta T Party